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ABSTRACT 



An overview of national and international research on 



school effectiveness is provided in this paper. Following an 
introduction, the first section examines some national studies, with 
a focus on the differences in empirical evidence for effective school 
factors. These studies are similar in that they are outcome-oriented, 
seek factors of school effectiveness, and combine theory with 
practice. A conclusion is that differences among countries are due 
not only to contextual factors, but to conceptual/methodological 
problens. The third section describes studies being conducted by the 
International Association for the Evaluation of Educational 
Achievement (IEA) and the Educational Testing Service (ITS). 
Conclusions are that secondary analysis of available international 
databases is useful to illustrate the differences among countries in 
terms of educational outcomes and determinants of educational 
effectiveness, and that international studies are strong in outcome 
measurement, but weak in determining factors at different levels. The 
concluding section suggests that school effectiveness research needs 
to develop a multilevel, multif actor framework that utilizes 
outcome-oriented measurement. Five tables and three figures are 
included. (15 references) (LMI) 
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Looking back to school-effectiveness research at this moment, one can conclude that 
different pieces of educational research in the past contribute to school-effectiveness. The 
start of school-effectiveness as a movement that influenced research-policy and educatio- 
nal practice is definately related to the research by Brookover (1979) and the combination 
of research and improvement by Edmunds. In fact, school-effectiveness as a movement 
within education was a more optimistic reaction to the pessimistic ideas and opinions 
about the contributions of education to outcomes, especially for disadvantaged groups. 
Based on the disappointing results of the special programs in the years before and the 
publications of researchers as Jencks and Coleman, the conclusion was drawn that schools 
and teachers did not matter and, in fact, education did not matter at all. Education was 
not an instrument to compensate for differences between students based on intelligence 
and social background. It was even difficult to find characteristics of teachers and schools 
that contribute to the quality of eduction, i.e. that could explain the differences in 
learning-results of comparable groups of students. Brookover and other researchers of that 
period proved that schools acnieved different results with comparable groups of students. 

Thus, schools do matter. Based on that conclusion, numerous research-projects started in 
order to find the characteristics of effective schools. Later on, these characteristics have 
been summarized in so-called factor-models; factors that contribute to educational 
effectiveness (at the beginning the school-effectiveness movement was strongly connected 
with the ideal of enlarging the equity in education. Later on, however, it turned out that 
effective schools were not just good for (did not just work for) disadvantaged groups of 
students, but also for the better students). These results were most of the time based on 
research in the United States, although, especially in the late seventies and eighties, 
school-effectiveness research was also carried out in other countries. Internationally, the 
research carried out by Rutter et al. (1979) and Mortimer et al. (1988) in the United 
Kingdom draw attention. Other countries with a considerable amount of research on 
school-effectiveness are The Netherlands, Canada and Australia. In 1988, the first 
international conference on school-effectiveness improvement was organized in London 
and the International Congress for School Effectiveness and Improvement (ICSEI) was 
founded. About 15 countries participated in this Congress and informed one another about 
the progress made in educational effectiveness. Later on, congresses were held in 
Rotterdam, Jerusalem, Cardiff and Victoria (British Columbia). In these meetings, the 
international attention for school-effectiveness was demonstrated. It also became clear, as 
was published in various research reports, that the factors that contributed to educational 
effectiveness did not work to the same extent in different countries. This was partly the 
result of the way the research was carried out and the conceptual models were used in 
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different countries, but also of the contextual differences between countries. It became 
clear that for the development of a knowledge-base on effective schools we need 
international research in which the contextual setting of the education within different 
countries can also be investigated. 

In the next section we will first deal with some national studies, espially to look at the 
differences in empirical evidence for so-called effective school-factors (section 2). In the 
following section (3), attention will be given to the international studies. Based on the 
conclusions about the flaws and strengths of these kinds of research the final section (4) 
will sketch the requirements for the future research-agenda in this field. 



2. National studies on effective schools 

In the United States in the seventies and eighties quite a lot of research was carried out 
into the characteristics of effective schools, and how to improve schools according to 
these characteristics. After a while, factor-models were developed that were based on this 
research, suggesting that by this specific set of factors the effectiveness of schools could 
be explained (and improved), The most famous factor-model is Edmund's 5 -factor model, 
especially because it was restricted to only 5 factors by which education in general and 
schools in particular could be improved to get better results and to contribute to educatio- 
nal equity. These 5 factors were: 

- Strong educational leadership 

- Emphasis on basic skills 

- A safe and orderly climate 

- Frequent evaluation of the pupil's progress 

- High expectations of the student's achievements 

Although almost every presentation about effective schools in the United States starts with 
the confession that the presenter does not believe in the 5-factor model anymore, this 5- 
factor model heavily influenced research and educational practice in the United States. 
Based on these factors improvement-programs were developed, and especially the training 
of principals into educational and instructional leadership were organized. The presenta- 
tions at the A ERA of the last couple of years in the field of school-effectiveness deal with 
the results of these training-programs for principals (without even phrasing the question of 
the effectiveness later on in the schools of these well-trained principals in terms of 
student-outcomes). With more or less creativity and imagination the current movement of 
restructuring schools can be connected with these 5 factors, because what they aim at is 
to improve the educational practice within schools by developing and improving the 
curriculum of the schools and the organization (some of the factors are connected with 
organization, others, such as the objectives and the evaluation, with the idea of the 
curriculum and the instruction-process). 

Edmunds claimed that the 5-factor model is based on his own and others' research. One 
of the latest reviews of the research-literature on school-effectiveness was carried out by 
Levine and Lezotte and published in Unusually Effective Schools (1990). It becomes clear 
from their review that it is not possible anymore to provide easy recipies for educational 
effectiveness; it is not possible to improve a few components of education to get better 
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results. On the contrary, many factors all contribute a little to improvement of education. 
The list of Levine and Lezotte is a mixture of factors on different levels within the school 
connected with all the participants in the educational process as shown in the following 
enumeration of factors: 

* * Productive School Climate and Culture 

* Focus on Student Acquisition of Central learning Skills 

* Appropriate Monitoring of Student Progress 

* Practice-Oriented Staff Development at the School Site 

* Outstanding Leadership 

* Salient Parent Involvement 

* Effective Instructional Arrangements and Implementation 

* High Operationalized Expectations and Requirements for Students 

* Other Possible Correlates 
(Levine and Lezotte, 1990, p. 10) 

Each of the factors ('characteristics' according to Levine and Lezotte) is a container of 
subordinated characteristics, e.g. 'Effective Instructional Arrangements and Implementati- 
on' is more or less a combination of effective teaching characteristics. It contains: 

- Successful grouping and related organizational arrangements 

- Appropriate pacing and alignment 

- Active/enriched learning 

- Effective teaching practices 

- Emphasis on higher order learning in assessing instructional outcomes 

- Coordination in curriculum and instruction 

- Easy availability of abundant, appropriate instructional materials 

- Classroom adaptation 

- Stealing time for reading, language and math 

The fact thatLevine and Lezotte include a special list for low SES schools (based on 
Hallinger and Murphy, 1986) points to the progress in research on quality and equity in 
education and the change of opinions with respect to these topics. I have some doubts 
about the quality of the research on which this list is based, but there is at least some 
empirical evidence for the factors. My guess is, however, that the lists will not have as 
much impact as the 5-factor model, because for practitioners, the 5-factor model is as 
much as needed and as few as possible (and connected to that, we have got to do so much 
already to get better results and a better school). 

As one can expect, the idea of effective schools and especially the factors connected with 
effectiveness draw attention in other countries. In fact, at the time the results of American 
researchers were published, in other countries , such as the U.K. research took place to 
investigate differences between schools with respect to educational outcomes in a broad 
area like learning results, delinquency-rates and so on (Rutter et al., 1979; Rutter, 1983). 
Later on, a study by Peter Mortimore and others on junior schools draws attention 
(Mortimore, Sammons, Stoll, Lewis and Ecob, 1988). Mortimore and his co-workers 
came to the conclusion that 12 kev-factors that were under the control of the head and the 
teachers were crucial for effectives. These key-factors were: 



1. Purposeful leadership 

2. The involvement of the deputy-head 

3. The involvement of teachers 

4. Consistency among teachers 

5. Structured lessons 

6. Intellectually challenging teaching 

7. Work-centered environment 

8. Limited focus within sessions 

9. Maximum communication between teachers and pupils 

10. Record-keepLig 

1 1 . Parental involvement 

12. Positive climate 

(Mortimore, Sammons, Stoll, Lewis and Ecob, 1989) 

Comparing the key-factors by Mortimore and others with the factor-model from Edmunds 
and the list of variables influencing school-effectives by Levine and Lezotte, we can see 
some congruency between them, but also differences. In the long list by Mortimer et al. 
and the one by Levine and Lezotte, the factors of Edmunds do not appear. High expecta- 
tions, emphasis on basic skills, safe and orderly climate do not appear in the same way in 
the studies by Mortimore and Edmunds. Educational leadership gets a very specific 
content in the U.K., with the deputy-head who plays a major role in the effectiveness of 
schools. In the eighties quite a lot of research with respect to school-effectiveness took 
place in The Netherlands (for a review, see Creemers, 1989 and Scheerens, 1990). In this 
research could be proved, and that was the positive part of the conclusions, that between 
12 and 18 percent of the variance could be explained by school-factors. The negative part 
of it was that most of the factors included in the studies derived from research in the 
United States, did not explain very much. 

Scheerens and Creemers provided an overview of Dutch studies so far and the significant 
relationsships found (see table 1). In comparison with American and British research, 
some of the results are striking, e.g. for educational leadership, that only in one study 
empirical evidence could be found. More empirical evidence was available for the factors: 
orderly climate, frequent evaluation, achievement-orientation. The earlier Dutch studies 
could, like the American ones, be criticized. According to Ralph and Fenessey (1983) 
these studies could be criticized for methodological flaws and technical research-pro- 
blems, but also later studies in which these problems were solved, show comparable 
results. 



Table 1 



Table LI 

Overview of Dutch Effectiveness Studies 



Indicator 



Author 



General 
measure 
of school- 
climate 



Educational Orderly Basic High Frequent 

leadership climate skills expectations evaluation 



Direct Achievement 
instruction orientation 



Meijncn, 1985 

Marwijk-KooyJ985* 

Hocvcn van Doomum. 

JungWuth, 1987 

Stoe!,1986 t 

Bo$ker,Hofman, 

1987 

Brartdsma.StoeL 
f9K7 a 

Vermeulen, 1987 
Tefscr, 1985* 
Van cer Wolf. 1985 
Brandsma, Knuver. 
1988 

VanderWerf, 
Tcsser, 1989 
De Jong. 1988 



" Secondary schools; unmarked Primary schools; + means significant positive relationship with effectiveness indicator. 



Source: Scheerens and Creemers (1989, p.695) 



So, between countries the results of educational research show quite a lot of differences in 
the factors that contribute to school-effectiveness. Some of the differences can be 
explained by the differences in context and differences between the educational systems of 
the countries, or, in other words, it is at least interesting to look at factors in the societal 
and educational systems of the countries. However, it can also be the result of conceptual 
and methodological problems in the research on school-effectiveness within countries or 
between countries. Before we deal with the explanations for the differences in the results 
between countries we shall first sum up the points these studies have in common: 

1. All the studies are outcome-oriented, i.e. the ultimate criteria for educational effects 

are learning outcomes (that what should be achieved by education in schools). 
Educational improvement within the paradigm of school-effectiveness and school- 
effectiveness research has as its point of departure that schools have to deal with 
learning and learning-results. After that statement, there are a lot of differences: 
sometimes learning-outcome is defined as 'school-career' (the transition from one 
grade to another or learning-results, with correction for differences in entering- 
behaviour (added value). 

2. Educational effectiveness-research is looking for factors, variables and determinants 
for educational effectiveness. Outcome is important, but the task of education and 
educational research is to look for factors that can explain differences in outcomes. 

3. Within the framework of factors it is important that school-effectiveness contains 
effectiveness of the school as an organization and instructional system, but also what 
happens on the classroom-level or the levels above the school. Especially later on, 
research moved from school -effectiveness to educational effectiveness as a combinati- 
on of contextual, school- and instructional effectiveness. 

4. School-effectiveness theory and reserach is moving from a school-oriented theory and 

research-programs to a program about "what works in education" and "why". 

5. Because of the fact that educational effectiveness research deals with what works in 

education and stems from the idea that education should contribute to learning results 
of all children (including disadvantaged groups), there has always been a strong 
relationship between research and theory on the one hand and practice, improvement 
of education and educational policy-making on the other. Educational effectiveness 
provides, therefore, the possibility to combine theory, research and improvement. 

Recent research has demonstrated that the influence of school-factors decreases when 
instructional factors are introduced into the analysis, while other school- factors still have 
a unique contribution to the explanation of variants in outcomes. Based on the results, we 
expect factors on the instructional level to contribute to effectiveness (see Levine and 
Lezotte, 1990, Mortimore, 1989 and Creemers, 1992). the evaluation of student-learning, 
as a part of the instruction but also on the school-level (see Levine, Lezotte, Mortimore 
and Scheerens), the organization of the management of the school (educational leadership, 
see Levine, Lezotte and Mortimore). The results of Dutch research could be explained by 
the fact that the principal in Dutch schools does not have the same power and responsibi- 
lity as principals in other countries, and high expectations about student-learning on the 
school-level, the mission of the school are connected with a restricted set of objectives. 



The research itself in different countries shows a lot of differences and is in fact no 
confirmation of the results found in the U.S. -research. In fact, within countries and 
between countries we found a lot of contradictory results, so the conclusion about what 
works in education is 2 difficult one. Apart from the above-mentioned contextual 
differences between countries, there could be other reasons for the observed discrepancy: 

1. An important one is the already mentioned methodological, technical and statistical 
flaws in the research that was carried out. For example, in the United States school- 
effectiveness research started with so-called outlier studies. Therefore, we do not 
know what happens in schools between the outliers. To be able to generalize the 
conclusions, we need the full spectrum of schools. 

2. Sometimes the set of outcome-measures is quite restricted: school-career or learning- 
results in one instead of the learning-results in several school-subjects. 

3. What goes on in schools, the factors for effectiveness of education are often measured 
in an unsatisfactory manner. Caused by means of a survey instead of by observation, 
only a few components of the educational process are high-lighted. 

4. The earlier studies were just correlational studies instead of studies explaining 
variance. 

5. School-effectiveness research started looking for factors that could explain differences 
with no theoretical background. It was, in a way, quite empirical, just looking for 
correlations without any idea behind it about what actually happens in education. This 
resulted in a number of conceptual problems, e.g. it was not clear what was a cause 
and what an effect, the factors were situated on several levels in education and 
instruction (e.g. the school-level, the instructional level and so on). Several authors 
(Reynolds, Scheerens and Creemers) make a plea for a better conceptual model that 
distinguishes more levels in education and more factors in a mutual relationship 
within the levels. Furthermore, the contextual nature of education should be taken 
into account. The basic model Scheerens and Creemers developed is provided in 
figure 1. Later on, both authors developed a further elaboration of this model (see 
figure 2: Scheerens 1990 and figure 3: Creemers 1992). 



Figure 1 
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Figure 2 



PROCESS INDICATORS OF SCHOOL FUNCTIONING 



inputs 

- teacher experience 

- per pupil expenditure 

- parent support 



context 

- achievement stimulants from higher administrative levels 

- development of educational consumerism 

- "co-variables" like school size, student-body composition, 
school category, urban/rural 



T 



[process | 
^school level I 
. - degree of achievement oriented policy . 
■ - educational leadership 1 
| - consensus, cooperative planning of J 
I teachers i 
I - quality of school curricula in terms of J 
I content covered, and formal structure I 
|- orderly atmosphere J 



__3Z 

I classroom level 



n 



j- cimc-on-task (including homework) J 
. - structured teaching 

I - opportunity to team I 

I - high expectations of pupils' progress J 
. - degree of evaluation and monitoring or 

I pupils* progress | 

| - reinforcement j 



outputs 
student 

achievement, 
adjusted for: 
. previous 

achievement 
. intelligence 

SES 
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Figure 3 
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3. Intern ational studies 

The national studies do not provide information that makes it possible to draw conclusions 
about the generalizibility about the characteristics or factors for educational effectiveness. 
Considering the studies, it is plausible that some factors are more general than others, but 
even when the research carried out does not show the flaws of the past and uses a better 
conceptual framework, some contextual factors, such as consensus, may influence the 
results, i.e. may influence the way in which factors contribute to effectiveness. The 
above-mentioned example concerning educational leadership is useful in this respect. The 
very consensus-oriented democracy in The Netherlands and the fact that the status of the 
principal is more 'primus inter pares' (the first among equals) instead of a person in 
charge of the management of a school and the instructional management can explain the 
fact that educational leadership does not work in The Netherlands. 

Aside from other objectives for comparative research, such as finding out what the results 
of the national educational system are compared to the outcomes of educational systems in 
other countries, from the point of view of a theory about the educational effectiveness are 
more more reasons to carry out international comparison in order to provide information 
about which factors travel from country to country and, more specific, what contributes to 
effectiveness in the context of schools. Even when schools are places to teach and learn 
and the processes going on there are quite comparable, the condition' for schooling and 
education are different from country to country. International comparative research 
reveals this kind of information. 

The studies by the International Association for the Evaluation of Educational Achieve- 
ment (IEA) are especially carried out to provide insight in educational outcomes of 
participating countries. IEA started in the late fifties and has since then carried out studies 
in language, arts, mathematics, science, classroom-environment, educational technology 
and is now again preparing evaluation-studies in mathematics and science. The studies 
itself are very careful and provide an enormous amount of data, not just concerning 
educational outcomes (the main focus of the studies) but also about the educational 
process to achieve the goals. 

A disadvantage of the IEA is the fact that these studies require quite a long period of time 
(up to 10 years), which is due to the negotiations with different countries about funding, 
negotiations within the project-team itself concerning the test-items, the methods of data- 
collecting and the analysis per country and between countries. However, the advantages is 
that it provides data-sets that are quite unique and provide a great source for secundary 
analysis. It has turned out that most of the time only the first analysis, the overall 
analysis, with all the countries involved received the most attention from educational 
practitioners and policy-makers ('Where is our country in the international competition?'). 
But, further analysis about what goes on in education with respect to specific school- 
subjects and the way it contributes to outcomes and maybe can explain some of the 
differences in the participating countries is more interesting, at least to educational 
researchers and educational theorists. 

At this moment, we have a competitive international study going on, namely "Internatio- 
nal Assessment of Educational Progress" (IAEP) carried out by the Educational Testing 
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Service (ITS). The ITS carries out the national assessment program in the United States, 
and the translation of the English texts developed in the U.S. are made avaible for other 
countries, which all have their own specific reasons to collect the data. The analysis and 
the international comparisons are, again, performed by the ITS, and they do it fast. On 
me other hand, however, there is a great disadvantage, namely the cultural bias of the 
tests done in the U.S.A. In addition to this, international study does not provide much 
insight in what works in education, or, in other words, which factors contribute to 
educational outcomes in different countries and between countries. Because of the fact 
that data-sets are available about the context, the background of students and teachers, the 
teaching-processes and schools, IEA data-sets are interesting for secundary analysis with 
respect to educational effectiveness. 

Up to now, only a few studies available in the secundary analysis are carried out on IEA 
data-sets in which different countries are engaged. An example of secondary analysis is 
the study carried out by Lockheed and Komenan on teaching quality and student achieve- 
ment in Africa. The study of Lockheed and Komenan was directed towards Nigeria and 
Swaziland, and they made use of data-sets of the second international mathematics study. 
Table 2 provides us with an overview of the variables, names, descriptions, the means, 
and standard deviations for Nigeria and Swaziland. Making use of the HLM technique, 
specifying different models they conclude that the achievement of students in both 
countries was significantly affected by the school and classroom in which they were 
enrolled, once effects of family characteristics were controlled. However, the specific 
school and classroom variables accounting for these differences were not the same in both 
countries. Substantively, effective teaching practices in one country would be entirely 
ineffective in another one. For example, in Nigeria mathematics classes who spent more 
time listening to the teachers introduce and review mathematics outperformed those who 
were less exposed to direct instruction. The same result was not found in Swaziland. 
However, teaching time spent monitoring and evaluating student performance was 
positively associated with achievement in Swaziland, while it had no effect on achieve- 
ment in Nigeria. A significant proportion of variance in student achievement, 24 % in 
Nigeria and 16 % in Swaziland, was attributable to between classroom and school factors. 
School size, class size, and length of the schoolyear have no effect on student achieve- 
ment, but teaching processes were important in both countries. This means that not 
teacher quality but teaching practices are important in education. 
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Table 2 



Variable Names, Descriptions, Means, and Standard De viations (in parentheses) for Nigeria and Swaziland 



Variable 



Description 



Nigeria 

(/V«70Q) 



Swaziland 
(JV-5S7) 



SCORE 

Background 
YSEX 

YAGE 

YFPROF 

YPERCEV 

YMOREED 

YMOT1V 

YPARSUP 

RURAL 

School 
ISENROL 

ISDAYSYR 

SINGMALE 

SINGFEM 

TeacherlCtass 
TNSTUDS 

TEX PTC H 

TEDMATH 

AVYFPROF 

Teaching process 
TADMNTASK 

TfNSTASK 

TMONEVAT 

TLISTL 

TSEATL 

TPERSMAT 

TPUBMAT 

OTL 



Student's core test score 

Student's sex (0 = male; I « female) 

Student's age in months 

J » Father has professional occupation 

Student's self-perception of mathematics ability 

Years more education expected 

Motivation to work hard and do well in mathematics 

Perceived parental support 

I = School in rural area 

School size (number of students enrolled in the school) 
Length of school year in days 
1 All male school 
I ~ All female school 

Class size (number of students enrolled In dass) 
Teacher's experience (in years) 
Semesters post-secondary mathematics education 
Percentage of professional fathers in each dass 

Weekly minutes for routine administration and 

maintaining order 
Weekly minutes for eaplainiftg ne* material and reviewing 

old material 
Weekly minutes for testing and grading 

Weekly minutes students spent listening to whole dass lectures 

Weekly minutes students spent at seat or blackboard 

Use of personally produced teaching materials 

Use of commercially published teaching material 

Opportunity to learn (number of test questtora covered by 
teacher during current academic year) 



14.36 


12.92 


(5 80) 


(6.94) 


0.24 


0 58 


(0.43) 


(0.49) 


196.20 


185 83 


(20.84) 


(20.30) 


0.21 


0.13 


(0.41) 


(0.34) 


3.18 


3.91 


(1.19) 


(130) 


3.64 


3.26 


(1.00) 


(099) 


It.t. 


4 18 




(152) 


3.66 


n.a. 


(1.53) 




0.22 


031 


(0 41) 


(0.46) 


1054.2 


374.23 


(354 5) 


(139.86) 


188.03 


191.02 


(14.04) 


(0.72) 


0.41 


0.03 


(0.49) 


(0 18) 


0.10 


0.14 


(0.30) 


(0.35) 



34.92 
(15.05) 

8.04 
(9.10) 

3.61 
(1.44) 

0.21 
(0.17) 



70.46 
(63.68) 
117.22 
(106.36) 
162.04 
(115.22) 
37.70 
(33 60) 
42.28 
(38.03) 
5.51 
(0.84) 
8.76 
(166) 
11.40 
(10.95) 



38.15 
(673) 

4.78 
(4 73) 

2.97 
(2.75) 

0.12 
(0.12) 



30,60 
(28.15) 
78.38 
(45.19) 
138.21 
(39.14) 
36.17 
(27.47) 
57.79 
(44.59) 
4.55 
(114) 
9.57 
(1.68) 
10.41 
(5-38) 



Source: Lockheed and Komenan (1989, p. 100) 
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Another study is carried out by Scheerens, Vermeulen en Pelgrim about the gereralizibi- 
lity of the instructional and school -effectiveness indicators across nations. They used the 
second maths-study data-set and a list of 15 predictor variables mentioned in table 3. 
These variables are not specific to educational effectiveness, but are used as proxi- 
variables for educational effectiveness factors. In that respect they are questionable, e.g. 
'opportunity to learn' is defined in IEA-studies as 'the indication by the teacher or the 
student, whether are not the items of the test are covered by the instructional process'. 
But there is no information at all about 'opportunity to learn' or the time that is spent on 
the specific item or topic. The same holds for 'expectations': this is rarely a definition of 
'high expectations', because high expectations in the educational effectiveness movement 
means 'the standards the teacher and the student have for their teaching and learning'. In 
the analysis it turned out that there were differences in the estimates of variance explained 
by schools and classes between the countries. In a country like Sweden there is no school- 
variance component, but a rather large variance component on the classroom-level. This 
is probably a result of the way in which the Swedish society is organized: no differences 
between catchment-areas and the centralists way of organizing schools. However, even in 
Sweden differences still exist or arise behind the classroom-door. 
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Table 3 



List of 15 predictor variables: 



— Pupil characteristics 

father s occupation (yffocci), 
father's education (yfeduc), 
— Teacher characteristics 

experience as a mathematics teacher (in years — iexpmth), 
time spent on keeping order (in min per week — tordert), 
time spent on teaching (in min per week — tlisst). 

— Opportunity to learn 

items to test covered in tuition (toil). 

— Expectations 

pupil's expectation of the number of years he/she wilt follow formal education 
(ymoreed). 

estimate by teacher of the .lumber of pupils who belong to the top in mathematics 

(ttopY 

— Instructional characteristics 

total time (h <urs) spent on homework (yahwki), 
the use of published tests (tpubst), 
the use of teacher made tests (towntst). 

— School characteristics 

the number of women teachers in mathematics {ssommf), 

the number of men teachers that teach only mathematics (sail mm). 

the number of meetings of mathematics teachers (snteet). 

— Contextual characteristics 

degree of urbanization of the school area (saera) f 
class size (klgrt). 



ERIC 



Source; Scheerens and Creemers (1989, p. 792) 
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The high estimates of the school-variance components for Belgium and The Netherlands 
are indicative of a strongly differentiated organization of the national school-system (see 
table 4). Table 5 provides the predictor variables with a significant positive or negative 
association with mathematic achievement. The predictive variables that showed the 
strongest and most consistent positive association with mathematics achievement are 
educational expectations as expressed by pupils and teachers. Further predictor variables 
that show a consistent positive association with achievement are total time spent on 
homework and opportunity to learn. The results also indicate that in most cases the 
selected sets of predictor variables operated more on a classroom-level than on a school- 
level. 



Table 4 



Estimates of the Variance Explained by Schools and Classes 



Classroom variance 
ciMnponenf 



School variance 
component 



15 Belgium (Flemish) 

16 Belgium (French) 

22 Canada ( British Columbia) 
25 Canada (Ontario) 

39 Finland 

40 France 

43 Hong Kong 

44 Hungary 
51) Israel 
54 Japan 

59 Luxembourg 

62 Netherlands 

63 New Zealand 
72 Scotland 

76 Sweden 
79 Thailand 
81 U.S.A. 



I8(.17) 
.45(41) 
17(16) 



.50 ( 48) 
.64 (.62) 
.27 (.27) 
.09 ( 09) 
.002 



45( 42) 
34( 31) 
45( 45) 



22< 21) 



,29(.29) 



.06(.O5) 
.51 (.50) 
30(27) 
10(08) 
.08 (.07) 
.15(.!5) 
.67 (.66) 
.01 ( 0O4) 
.12(05) 
.00 



.46( 45) 



.39(38) 
.10(.09) 




Source: Scheerens and Creemers (1989, p.794) 
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Table 5 



Predictor Variables with significant Positive (+) or Negative (-) Associations (5% Level) with Mathematics Achievement, when the Variance Component 

Model is Analyzed by Means of the VARCL-Programme 
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Note: School and Jassroom predictor variables are corrected for father's occupation or father s education — when a predictor variable was not measured j 
in a country this is indicated by the letter m. I 
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Source: Scheerens and Creemers (1989, p.796) 
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It is clear that it is very useful to carry out secundary analysis on available international 
data-sets. This is not just to show differences between countries with respect to educatio- 
nal outcome., but also with respect to the detenrinants for educational effectiveness. 
These determining factors can be found on the classroom- as well as on the school-level, 
but there is some indication that they can especially be found on the classroom-level. The 
international secundary analysis provides some evidence for factors and also for the 
contextual differences that influence educational outcomes. However, international 
studies, and this is not surprising because they were developed that way, are especially 
strong in outcome-measurement, but weak in the way they measure factors that contribute 
to educational effectiveness on the instruction -, school - and contextual level. This is not 
just the case for the instruments, but also for the way the data are collected. For these 
large-scale international studies, the only way to collect data on process-variables is by 
means of surveys, and in several countries teachers and principals are perfectly aware of 
how to answer these kinds of questions. 



4 - The researcher's agen da for future research on educational effectiveness 

Based on the analysis we carried out in the sections 2 and 3 we have to conclude that for 
international study we need the development of a model, a theory, a conceptual frame- 
work for educational effectiveness. This framework should be multi-level and at least 
contain a classroom-, school- and contextual level. It should also be multi-factor, so that 
on each level we find all factors that contribute to the effectiveness of a specific level. 
The multi-level and multi-factor approach should include the possibility that we can see 
the same factors between the levels, e.g. what curricular conditions are available for the 
method on the classroom-level. The framework should also be contextual, taking into 
account the contextual differences. 

With respect to the instrumentation: more attention should be given to outcome-measures, 
not only cognitive outcomes but also social ans affective outcomes. Special attention 
shoud be given to the process-measures on the different levels. The only way of getting 
information on the classroom-level is by means of observation. On the level above the 
classroom-level, the school- and the educational context, the only way of collecting the 
information is by means of surveys. As is clear from the multi-level and multi-factor 
conceptual framework, the statistical procedures for analysis should follow this frame- 
work. This means that the adequate technique is a multi-level analysis connected with 
causal analysis. 
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At this moment, we do not have a theory and knowledge-base that provide enough 
empirical evidence to build on. That means that in a research program for international 
studies on educational effectiveness we have to combine survey research on a large 
number of students and schools with (in)depth-analysis of schools in countries. In that 
respect, the so-called outlier-studies which were used in the past to obtain information 
about what factors distinguish effective and non-effective schools are back again. By 
means of this (in)depth-analysis, research can be carried out into the schools in transition, 
going from effective to non-effective and from non-effective to effective. That also 
includes, aside from (in)depth-analysis and (in)depth-studies and survey-research, 
longitudinal studies. The International School-Effectiveness Research Program (ISERP), 
will develop a series of studies will be developed according to the above-mentioned 
requirements. 
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